The present invention relates generally to human interfaces of any application program for computers and more particularly to a metaphor environment control system which displays a plurality of metaphors representative of various functions of the application program and presents relevant functions according to a user's operations.
Important for the human interfaces of computer systems is to build system images easily appreciable by users. In recent years, it has often been pointed out that metaphors are efficacious for building these system images and various efforts have been made to implement metaphors.
However, it has generally been indicated that metaphor interfaces have the following problems.
(1) Mapping between a function group and metaphors provided by applications is still less than perfect. PA1 (3) In some cases, metaphors that the designers consider relevant may be irrelevant to users. PA1 (4) New functions cannot be expressed by known metaphors alone. PA1 (1) the master is asked about the program upon receiving from the channel (slave) a report that the user has given instructions on a certain channel button, PA1 (2) the program (slave) is required to be represented upon receiving from the master a request on the representation of a certain program, PA1 (3) the channel (slave) is asked to give a set of default channel numbers upon receiving from the power source (slave) a report that the user has turned it on, PA1 (4) the default program is represented upon receiving from the power source (slave) a report that the user has turned the power source on, PA1 (A) The user intends to do news retrieval by a newspaper independent from the TV set metaphor. PA1 (B) The user intends to make use of the newspaper's television program guide so as to switch program on the TV set metaphor. PA1 (C) The user intends to review the news being now represented on the TV set metaphor in the newspaper metaphor format. PA1 an operational model for storing a user's operational information, PA1 a world model for storing the internal states of a plurality of metaphors to be manipulated, PA1 a display model for storing what states said plurality of metaphors are displayed in, PA1 an operation/display control block for rewriting said operational model depending upon the user's operation and changing the representations of said metaphors pertinently depending upon a change in said display model, PA1 an operational model/world model conversion rule for rewriting pertinent portions of said world and display models, taking a chance of a change in said operation/display control block for said operational model, PA1 a plurality of causal rules for rewriting other portion of said world model and a pertinent portion of said display model, taking a chance of rewriting said world model, and allowing one world model to request to launch a plurality of rules, and PA1 an ambiguity resolution block for selecting any one of said plurality of causal rules, when they are launched.
(2) A single metaphor fails to cover many functions.
A leading cause of the problems involved with such metaphor interfaces appears to be due largely to the fact that conventional metaphor interfaces are implemented without taking into account the mechanism of how humans understand metaphors.
In general, the mechanism of how humans understand metaphors is modeled by partial mapping of the structurally modeled object to be recognized, as represented by Gentner's structure-mapping theory.
In the case of metaphor interfaces, this partial mapping takes place between a task domain provided by an application and an object represented as a metaphor interface. There are two subjects who perform this mapping: one designer and one user. The problems mentioned above arise when the two subjects make the mapping differently.
Applicant has already come up with a system incorporating a television metaphor environment which allows a user to receive information in the sense of seeing a TV news program (Japanese Patent Application Nos. Hei. 2-154724 and Hei. 2-154725). The general construction of that system is diagrammatically illustrated in FIGS. 1A and 1B.
An information-presenting system incorporating the TV metaphor environment is built up of a computer 21 (e.g., Personal Computer PC9801RX made by NEC), an electronic voice utterance device 22, a mouse 23, a modem 24 and a telephone line 25 as hardware and a script interpreter operating on, e.g., MS-DOS and a script description file giving commands to this, as illustrated in FIG. 1A. This system achieves an environment allowing a user to see news programs by operating the TV metaphor through the electronic voice utterance device 22 and display 26. Data available to this end, for instance, include not only "Mainichi Sinbun On-Line News Flashes", "Mainichi Sinbun On-Line Local News", "The Nationwide Weather Forecast", all served by NIFTY-Serve, and data acceptable from other personal computer networks, but General-purpose data as well. An example of information presentation on the screen is illustrated in FIG. 1B.
The functionality needed Generally for the information presentation system above are largely broken down into a portion constituting the TV metaphor environment and communication function.
The television metaphor environment is made up of four metaphors: the TV set, VTR controller, news program and program guidance, which are specified by independent scripts.
In the TV set and VTR controller scripts, it is described how the states in the TV set and the program should change when those metaphors are operated. For instance, state changes that occur when the user presses the power switch, volume or channel button is described. In the news program script, the flow of a news program and its rules are described regarding how the data of a personal computer network, e.g., NIFTY-Serve, is constructed and presented as a news program, for instance, how the input data is processed and presented, and in the program guidance script, the structure and rules of program guidance are described. The scripts of these metaphors are stored in a script file, which is read and operated by the script interpreter.
The communication functionality is specified by a script describing how the communication functions of the script interpreter (that are able to handle selected data input from RS232C as an event and transmit the data) communicate with the personal computer network, e.g., NIFTY-Serve.
In the information-presenting system, the TV set 31 and the VTR controller 32 are always displayed on the display screen, as illustrated in FIG. 1B. And, with some exceptions, the power switch 33, channel button 34 and volume button 35 of the TV set 31 and the respective buttons (36-41) of the VTR controller 32 are operable on every scene, and operations on these are called the basic operations. In addition, some metaphors appearing on a CRT 42 can be operated on particular scenes.
The television metaphor environment mentioned above may be designed to present information in newspaper format suitable for browsing or, in the alternative, in TV program format suitable for grazing. At this time, the rules for interpreting the user's operation are left to the designer, but in consideration of plural users and a variety of applications, it is difficult to set down those interpretive rules unequivocally. For instance, the operation that the user presses the fast-forward button of the VTR may be taken as an action of turning the pages of a newspaper in the context that representation is in newspaper format and as an action of fast picture search in the context that representation is in television program format. Furthermore, the operation that the user presses the play button of the VTR may be interpreted as a request for still pictures to be moved in the context that the user sets an eye on a photograph in a certain news item. Possibly, a metaphor of a clock may be used for fast picture search as well. Thus, the VTR's fast-forward button corresponds to plural meanings of "turning the pages" and "fast picture search", and so this is called a "equivocal metahor". Fast picture search, on the other hand, corresponds to the metaphor of the fast-forward button of the VTR and the metaphor of the watch, and so these are called "synonymic metapors". Ambiguities inherent in the equivocal and synonymic metaphors must be resolved so as to follow the user's context, knowledge, intention and preferences.
Applicant has also come up with a metaphor environment control mechanism which enables adaptive actions for an individual user to be implemented from the same condition and event (Japanese Patent Application No. Hei. 2-154260).
Referring to FIG. 2, rule bases 51-1 . . . 51-n set down rules regarding status transition for each metaphor, and a working memory 54 stores information about relations among the metaphors as well as status transitions and other states of the metaphors. An input/output control means 53 controls inputs and outputs between a display, a mouse and a voice utterance device, and a function call means 55 calls a certain function of the application. A rule base control means 52 is operable to rewrite the information stored in the working memory 54 with reference to the rule bases 51-1, . . . 51-n, place the function call means 55 under control and control the input/output control means 53, thereby implementing status transitions of the metaphors.
Referring first to how to construct and control the metaphor environment, one metaphor is constructed as one production system having several states, and the rules of the production system are such that states and events are defined as the left side and actions as the right side. The events include the user's operations, messages from other metaphors, timer events, etc. The actions have three kinds of control:status control, picture control and function call. Said status control gives rise to a change in a metaphor itself, said picture control triggers a change in metaphor representation and said function call calls a certain function of the application. Each metaphor has its own collective hierarchy, whose parent is called the master and whose children are called the slaves.
The television metaphor environment built as shown in FIG. 3, by way of example, is made up of a TV set metaphor, a newspaper metaphor, a VTR controller metaphor, etc., and objects presented to the user includes a TV set, a caster, VTR controller panel, a newspaper, a clock, and so on. These are constructed in the form of a collective hierarchy of objects as shown in FIG. 4. Not only can these objects be recombined depending upon the user's preferences, but what operations are done on the objects can also be interpreted in the user's context, enabling function call to be carried out flexibly.
The following is an example of information written as the rules of common knowledge regarding each metaphor.
For instance, what may become slaves in the TV set metaphor are the channel, volume buttons, power switch, program and other metaphors. The TV set metaphor can be in either an icon state or an open state, said open state being further broken down into on and off states. In addition, the on-state includes a multi-dimensional condition expressed by the program being represented, the channel number and the sound volume level. Among the rules to be written, there are, for instance,
and so on. The working memory has a master/slave hierarchy structure of each metaphor.
In most cases, the operation the user conducts on the metaphor environment may be interpreted as having several meanings. For instance, now let us consider the case where the user is opening the iconified newspaper metaphor by the mouse cursor in the television metaphor environment as mentioned above. Then, there are some possibilities, mentioned just below, about the user's task intention, i.e., what the user intends to do by opening the newspaper metaphor with the mouse cursor.
These ambiguities must be resolved on the basis of the context in which the user does their task and their preferences. In the ensuing description, reference will be made to an example in which such ambiguities are resolved by conflict or ambiguity resolution based on the variable CF and activation values referred to in the prior invention.
In the status slot area of each metaphor in the working memory, as shown in FIG. 5 information regarding the activation value is set in the television metaphor environment; information regarding the activation value, iconization and channel in the TV set metaphor; information regarding the activation value, representation/non-representation and news titles in the display metaphor; and information regarding the activation value and iconization in the newspaper metaphor.
The prior invention is designed to allow a redundancy of rules firing with respect to the same condition and event existing in the rule category of common knowledge written in one metaphor of the rule bases 51-1, . . . 51-n. In the case of the newspaper metaphor, for instance, all politics, economics, living and television program columns it has as slaves may be candidates for space to be filled when it is opened up. Then, the CF value is set for each rule to incorporate therein a mechanism enabling it to change in conformity to the individual users, so that the action of each metaphor can accommodate to the individual's preferences. Such personal adaptation, which holds for the priority order of the rules firing in one metaphor, is considered to make positive use of semantic relations among the metaphors, when the user performs her or his task on the metaphor environment. For instance, the space to be filled when the newspaper metaphor is opened is affected by what state the TV set metaphor on the same metaphor environment is in.
Such influences resulting from what kind of relations exist among the metaphors can be implemented by incorporating the activation value in each metaphor and letting the activation value of a certain metaphor have an influence on resolution of conflict with the rule of other metaphor. For opening the newspaper metaphor soon after the power switch of the TV set metaphor is being turned on, for instance, it is preferred that the activation value of the TV set metaphor be so increased that the information of that activation value can accelerate selection of the rule triggering the representation of the television column.
As mentioned above, the metaphor environment control mechanism according to the applicant's prior invention is designed to write the rule of each metaphor in the rule base and store status control information about the status of each metaphor in the working memory to place the status transitions of the metaphors under control, so that the CF value and the activation value of each metaphor, that are able to be set depending upon an individual user's preferences, can be incorporated in the rules and working memory, thereby implementing actions lending themselves well fit for the user's preferences.
In the metaphor environment control mechanism stated above, the action of the displayed object is written in terms of rules which are fied by the operation of the user regarding that action. If there is conflict of rules, then it can be resolved by the activation values and CF values. However, when the conflict or the ambiguity involve personal factors, a problem arises in connection with which rule is to be selected, because the activation values and CF values have been built in the rules of the individual metaphors. In other words, which rule is to be selected is determined by what priority order is assigned thereto and so the desired rule cannot be dynamically selected while the user is in operation, making it impossible to give the priority order depending on individual differences. Another problem with this mechanism--which makes no positive use of task flow (task context)--is that there is some limit on the accuracy of ambiguity resolution, because many users tend to patternize the flow of tasks regarding a certain routine application depending on their preferences.